A reinforcement learning approach for Virtual Network Function Chaining and sharing in softwarized networks

​© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Cognizant of the ease with which softwarized functions can be dynamically scaled according to real time resource requirements, and the fact that multiple services can have common VNFs in their chaining, this paper tackles the problem of cost effective deployment of online services from the perspective of sharing their VNF instances. First, we formally formulate the deployment problem under VNFs sharing. Secondly, given the NP-hard nature of the above problem, we propose a reinforcement learning (RL) algorithm capable of making intelligent placement decisions while considering multiple conflicting costs. Costs of transmission, VNF instantiation or energy consumption, among others. Thanks to the intelligence of the RL algorithm, simulation results show that the performance of the proposed algorithm is within a 14% margin and similar to an optimal solution in terms of request provisioning cost and acceptance ratio, respectively. Moreover, the algorithm results in more than a 20% and a 70% improvement in terms of request deployment cost and time compared to a state-of-the-art algorithm, and up to more than a 40% improvement in terms of cost compared to an algorithm that greedily minimizes the transmission or VNF activation costs.Postprint (author's final draft

Resource allocation and management techniques for network slicing in WiFi networks

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Network slicing has recently been proposed as one of the main enablers for 5G networks; it is bound to cope with the increasing and heterogeneous performance requirements of these systems. To "slice" a network is to partition a shared physical network into several self-contained logical pieces (slices) that can be tailored to offer different functional or performance requirements. Moreover, a defining characteristic of the slicing paradigm is to provide resource isolation as well as efficient use of resources. In this context, the thesis described in this paper contributes to the problem of slicing WiFi networks by proposing a solution to the problem of enforcing and controlling slices in WiFi Access Points. The focus of the research is on a variant of network slicing called QoS Slicing, in which slices have specific performance requirements. In this document, we describe the two main contributions of our research, a resource allocation mechanism to assign resources to slices, and a solution to enforce and control slices with performance requirements in WiFi Access Points.This work has been supported by the European Commission and the Spanish Government (Fondo Europeo de Desarrollo Regional, FEDER) by means of the EU H2020 NECOS (777067) and ADVICE (TEC2015-71329) projects.Peer ReviewedPostprint (author's final draft

Surveillance with alert management system using conventional cell phones

In the present paper we expose the development of a real-time monitoring prototype of human activities using a single cell phone equipped with some sensors. The monitoring system is reduced to the minimum expression; no special hardware is required as far as a conventional cell phone will monitor the user, classify the user activities with respect to a customized rule’s set, and finally trigger the corresponding alerts, when necessary.Postprint (published version

A deep reinforcement learning-based algorithm for reliability-aware multi-domain service deployment in smart ecosystems

The final publication is available at Springer via http://dx.doi.org/10.1007/s00521-020-05372-xThe transition towards full network virtualization will see services for smart ecosystems including smart metering, healthcare and transportation among others, being deployed as Service Function Chains (SFCs) comprised of an ordered set of virtual network functions. However, since such services are usually deployed in remote cloud networks, the SFCs may transcend multiple domains belonging to different Infrastructure Providers (InPs), possibly with differing policies regarding billing and Quality-of-service (QoS) guarantees. Therefore, efficiently allocating the exhaustible network resources to the different SFCs while meeting the stringent requirements of the services such as delay and QoS among others, remains a complex challenge, especially under limited information disclosure by the InPs. In this work, we formulate the SFC deployment problem across multiple domains focusing on delay constraints, and propose a framework for SFC orchestration which adheres to the privacy requirements of the InPs. Then, we propose a reinforcement learning (RL)-based algorithm for partitioning the SFC request across the different InPs while considering service reliability across the participating InPs. Such RL-based algorithms have the intelligence to infer undisclosed InP information from historical data obtained from past experiences. Simulation results, considering both online and offline scenarios, reveal that the proposed algorithm results in up to 10% improvement in terms of acceptance ratio and provisioning cost compared to the benchmark algorithms, with up to more than 90% saving in execution time for large networks. In addition, the paper proposes an enhancement to a state-of-the-art algorithm which results in up to 5% improvement in terms of provisioning cost.This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 777067 (NECOS project) and the national project TEC2015-71329-C2-2-R (MINECO/FEDER). This work is also supported by the " Fundamental Research Funds for the Central Universities " of China University of Petroleum (East China) under Grant 18CX02139APeer ReviewedPostprint (author's final draft

Estrategia de búsqueda de dispositivos basada en el historial de conexiones utilizando redes neuronales

La movilidad es una de las principales características de las redes de comunicación actuales y produce cambios en su estructura, que en muchas ocasiones no son advertidos por la totalidad de los dispositivos que la conforman, principalmente por la distancia entre los dispositivos y el rango de transmisión. Por ello, la comunicación entre dos dispositivos se convierte en un problema de enrutamiento, el cual se define como la búsqueda de trayectorias mediante una adecuada estrategia. Para abordar esta situación, planteamos una estrategia de búsqueda basada en el historial de onexiones del dispositivo móvil. Por naturaleza, una persona tiende a exhibir comportamientos repetitivos, por lo que, observando estos patrones conductuales podremos, con cierta certeza, ubicarlo en un espacio y tiempo específico. Tomando en consideración lo anterior, si utilizamos el historial de conexiones de un dispositivo y mediante las apropiadas herramientas stocásticas, se podría lograr una visión de la estructura de la red y de esta forma predecir la secuencia de dispositivos que formarían la trayectoria para la omunicación entre dos dispositivos. Utilizaremos como herramienta de predicción de secuencias redes neuronales y analizaremos su contribución en el diseño de un algoritmo de búsqueda de dispositivos móviles.Postprint (published version

Resource slicing in virtual wireless networks: a survey

New architectural and design approaches for radio access networks have appeared with the introduction of network virtualization in the wireless domain. One of these approaches splits the wireless network infrastructure into isolated virtual slices under their own management, requirements, and characteristics. Despite the advances in wireless virtualization, there are still many open issues regarding the resource allocation and isolation of wireless slices. Because of the dynamics and shared nature of the wireless medium, guaranteeing that the traffic on one slice will not affect the traffic on the others has proven to be difficult. In this paper, we focus on the detailed definition of the problem, discussing its challenges. We also provide a review of existing works that deal with the problem, analyzing how new trends such as software defined networking and network function virtualization can assist in the slicing. We will finally describe some research challenges on this topic.Peer ReviewedPostprint (author's final draft

A multi-domain VNE algorithm based on load balancing in the IoT networks

The coordinated development of big data, Internet of Things, cloud computing and other technologies has led to an exponential growth in Internet business. However, the traditional Internet architecture gradually shows a rigid phenomenon due to the binding of the network structure and the hardware. In a high-traffic environment, it has been insufficient to meet people’s increasing service quality requirements. Network virtualization is considered to be an effective method to solve the rigidity of the Internet. Among them, virtual network embedding is one of the key problems of network virtualization. Since virtual network mapping is an NP-hard problem, a large number of research has focused on the evolutionary algorithm’s masterpiece genetic algorithm. However, the parameter setting in the traditional method is too dependent on experience, and its low flexibility makes it unable to adapt to increasingly complex network environments. In addition, link-mapping strategies that do not consider load balancing can easily cause link blocking in high-traffic environments. In the IoT environment involving medical, disaster relief, life support and other equipment, network performance and stability are particularly important. Therefore, how to provide a more flexible virtual network mapping service in a heterogeneous network environment with large traffic is an urgent problem. Aiming at this problem, a virtual network mapping strategy based on hybrid genetic algorithm is proposed. This strategy uses a dynamically calculated cross-probability and pheromone based mutation gene selection strategy to improve the flexibility of the algorithm. In addition, a weight update mechanism based on load balancing is introduced to reduce the probability of mapping failure while balancing the load. Simulation results show that the proposed method performs well in a number of performance metrics including mapping average quotation, link load balancing, mapping cost-benefit ratio, acceptance rate and running time.Peer ReviewedPostprint (published version

Self-adaptive online virtual network migration in network virtualization environments

This is the peer reviewed version of the following article: Zangiabady, M, Garcia‐Robledo, A, Gorricho, J‐L, Serrat‐Fernandez, J, Rubio‐Loyola, J. Self‐adaptive online virtual network migration in network virtualization environments. Trans Emerging Tel Tech. 2019; 30:e3692. https://doi.org/10.1002/ett.3692, which has been published in final form at https://doi.org/10.1002/ett.3692. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.In Network Virtualization Environments, the capability of operators to allocate resources in the Substrate Network (SN) to support Virtual Networks (VNs) in an optimal manner is known as Virtual Network Embedding (VNE). In the same context, online VN migration is the process meant to reallocate components of a VN, or even an entire VN among elements of the SN in real time and seamlessly to end-users. Online VNE without VN migration may lead to either over- or under-utilization of the SN resources. However, VN migration is challenging due to its computational cost and the service disruption inherent to VN components reallocation. Online VN migration can reduce migration costs insofar it is triggered proactively, not reactively, at critical times, avoiding the negative effects of both under- and over-triggering. This paper presents a novel online cost-efficient mechanism that self-adaptively learns the exact moments when triggering VN migration is likely to be profitable in the long term. We propose a novel self-adaptive mechanism based on Reinforcement Learning that determines the right trigger online VN migration times, leading to the minimization of migration costs while simultaneously considering the online VNE acceptance ratio.Peer ReviewedPostprint (author's final draft

A multi-stage graph aided algorithm for distributed Service Function Chain provisioning across multiple domains

Network Service Providers (NSPs) envisage to support the divergent and stringent requirements of future services by instantiating these services as service chains, commonly referred to as Service Function Chains (SFCs), that are customized and configured to meet specific service requirements. However, due to the limited footprint of the Infrastructure Providers (InPs), these SFCs may have to transcend multiple InPs/domains. In this regard, determining the optimal set of InPs in which to embed the SFC request emerges as a complex problem for several reasons. First, the large number of possible combinations for selecting the InPs to embed the different sub-chains of the request makes this problem computationally complex, rendering optimal solutions only after long computations, especially in large scale networks, which is unfeasible for delay sensitive applications. Second, the unwillingness of InPs to disclose their internal information, which may be vital for making embedding decisions, usually implies the provisioning of single-domain solutions, which are unsuitable in this working scenario. In this regard, this paper first formulates the multi-domain service deployment problem under multiple request constraints, such as bandwidth or delay, among others. Then, due to the NP-hardness nature of the above problem, this paper proposes an algorithm that is aided by a multi-stage graph for computing a request embedding solution in a distributed manner, solving the problem in acceptable run-times. Results from different simulations reveal that the proposed algorithm is optimized in terms of acceptance ratio and embedding cost, with up to 60.0% and 88.7% improvements in terms of embedding cost and execution time, respectively, for some scenarios, in comparison with a benchmark state-of-the-art algorithm.Postprint (published version

Slicing with guaranteed quality of service in wifi networks

Network slicing has recently been proposed as one of the main enablers for 5G networks. The slicing concept consists of the partition of a physical network into several self-contained logical networks (slices) that can be tailored to offer different functional or performance requirements. In the context of 5G networks, we argue that existing ubiquitous WiFi technology can be exploited to cope with new requirements. Therefore, in this paper, we propose a novel mechanism to implement network slicing in WiFi Access Points. We formulate the resource allocation problem to the different slices as a stochastic optimization problem, where each slice can have bit rate, delay, and capacity requirements. We devise a solution to the problem above using the Lyapunov drift optimization theory, and we develop a novel queuing and scheduling algorithm. We have used MATLAB and Simulink to build a prototype of the proposed solution, whose performance has been evaluated in a typical slicing scenario.This work has been supported in part by the European Commission and the Spanish Government (Fondo Europeo de Desarrollo Regional, FEDER) by means of the EU H2020 NECOS (777067) and ADVICE (TEC2015-71329) projects, respectivel